Hypothesis Testing and \(t\)

I will work with the speed variable. On a basic level, a hypothesis is a specific value for the average here. For whatever reason, \(\mu=17\) motivates us. I consider all three distinct hypotheses though this is not entirely proper in the sense that any given application of hypothesis testing should have a clear hypothesis about the variable of interest rather than waffling amongst all of them as I will. I do so only for illustrative purposes. To begin, I need to specify both the hypothesis and the confidence level that I intend to use to evaluate it.

• Case 1: The hypothesis to advance is that 17 is the true average speed. The alternative must then be that the average speed is not 17; it could be smaller or larger than 17. I wish to evaluate this claim with 90% confidence.
  
• Case 2: The hypothesis to advance is that 17 or greater is the true average speed. The alternative must then be that the average speed is less than 17. I wish to evaluate this claim with 90% confidence.
  
• Case 3: The hypothesis to advance is that 17 or less is the true average speed. The alternative must then be that the average speed is greater than 17. I wish to evaluate this claim with 90% confidence.

Before getting to equations, something is important to note. The mean of the data, \(\overline{x}\), \(s\), the standard deviation, and \(n\) are all known from the data. There are then only two remaining unknowns, either \(t\) or \(\mu\).

⊕\[t=\frac{\overline{x} - \mu}{\frac{s}{\sqrt{n}}}\]

The \(t\) equation is given by:

⊕\[\mu = \overline{x} - t\left(\frac{s}{\sqrt{n}}\right)\]

One further algebraic manipulation before starting. Let’s solve for \(\mu\).

This fully defines the parts we require and the core problem because the data essentially gives us almost all of the unknowns. In some very basic way, conditional on data, only \(\mu\) or \(t\) from a given probability remain unknown.

Details of the Data

library(ResampleProps)
library(gt)
library(gtExtras)
Resampled.Means <- data.frame(Mean.Speed = ResampleMean(cars$speed, k=10000))
library(tidyverse)
Tab.Res <- cars %>% summarise(Mean = mean(speed),
                   SD = round(sd(speed), digits=3),
                   SE = round(sd(speed)/sqrt(n()), digits=3),
                   N = n(),
                   P05 = round(quantile(Resampled.Means$Mean.Speed, probs=0.05), digits=3),
P95 = round(quantile(Resampled.Means$Mean.Speed, probs=0.95), digits=3),
P10 = round(quantile(Resampled.Means$Mean.Speed, probs=0.1), digits=3),
P90 = round(quantile(Resampled.Means$Mean.Speed, probs=0.9), digits=3)
)
Table.Res <- Tab.Res %>% gt() %>%  
  tab_spanner(
    label = "Percentile of Simulated Means",
    columns = c(P05,P95,P10,P90)) %>%
  gt_theme_nytimes() %>% 
  tab_header(title = "Speed from cars: The Distribution of the Average")
Q05 <- quantile(Resampled.Means$Mean.Speed, probs=0.05)
Q95 <- quantile(Resampled.Means$Mean.Speed, probs=0.95)
Q1 <- quantile(Resampled.Means$Mean.Speed, probs=0.1)
Q9 <- quantile(Resampled.Means$Mean.Speed, probs=0.9)
Resampled.Means %>% ggplot() + 
  aes(x=Mean.Speed) + 
  geom_density() + 
  geom_vline(aes(xintercept=Q05), color="red") +
  geom_vline(aes(xintercept=Q95), color="red") +
  theme_minimal() + 
  labs(x="Resampled Average", title="Resampled Means", subtitle="Middle 90% in red")

The statistics of it all

• The mean of speed is 15.4 mph
• The standard deviation (SD) is 5.288 mph.
• There are fifty observations. [n=50]
• The standard error of the mean (SE) is 0.7478.

\[t = \frac{15.4 - 17}{\frac{s}{\sqrt{n}}} = \frac{-1.6}{0.7478}=-2.14\]

There are 2.14 standard errors between the mean that we obtain from the data and the hypothetical mean of 17; the value from the data is smaller, hence the negative.

The middle 90% of the simulated means range from 14.2 to 16.6.

Table.Res

Case 1: A Two-sided Alternative

library(radiant)
result <- prob_tdist(df = 49, plb = 0.05, pub=0.95)
# summary(result, type = "probs")
p1 <- plot(result, type = "probs") + 
  theme_minimal() +
  geom_label(aes(x=-1.677, y=0.25, label="t=-1.677")) +
  geom_label(aes(x=1.677, y=0.25, label="t=1.677")) +
  geom_label(aes(x=0, y=0.15, label="p=0.9"), size=2) +
  geom_label(aes(x=-3, y=0.075, label="p=0.05"), size=2) +
  geom_label(aes(x=3, y=0.075, label="p=0.05"), size=2) +
  labs(title="Critical Values", x="t")  +
  xlim(-4,4)
result2 <- prob_tdist(df = 49, lb = -2.14, ub=2.14)
p2 <- plot(result2) + theme_minimal() +
  geom_vline(aes(xintercept=-2.14), color="#099587") +
  geom_vline(aes(xintercept=2.14), color="#099587")  +
  geom_label(aes(x=-2.14, y=0.25, label="t=-2.14")) +
  geom_label(aes(x=2.14, y=0.25, label="t=2.14")) +
  geom_label(aes(x=0, y=0.15, label="p=0.963"), size=2) +
  geom_label(aes(x=-3, y=0.075, label="p=0.0187"), size=2) +
  geom_label(aes(x=3, y=0.075, label="p=0.0187"), size=2) +
  guides(size="none") +
  labs(title="Result", x="t") + 
  xlim(-4,4)
library(patchwork)
p1 + p2

Critical Values
P(-1.677 < X < 1.677)     = 0.9
1 - P(-1.677 < X < 1.677) = 0.1

Result: The p-value
P(X < -2.14) = 0.019
P(X > 2.14) = 0.019
P(-2.14 < X < 2.14)     = 0.963
1 - P(-2.14 < X < 2.14) = 0.037

t.test(cars$speed, mu=17, conf.level = 0.9)

## 
##  One Sample t-test
## 
## data:  cars$speed
## t = -2.1397, df = 49, p-value = 0.03739
## alternative hypothesis: true mean is not equal to 17
## 90 percent confidence interval:
##  14.1463 16.6537
## sample estimates:
## mean of x 
##      15.4

Case 2: A Lesser Alternative

library(radiant)
result <- prob_tdist(df = 49, plb = 0.1)
p1a <- plot(result, type = "probs") + theme_minimal() + geom_label(aes(x=-1.299, y=0.25, label="t=-1.299")) +
  geom_label(aes(x=0, y=0.075, label="p=0.9"), size=2) +
  geom_label(aes(x=-3, y=0.075, label="p=0.1"), size=2) +
  labs(title="Critical Value") +
  xlim(-4,4)
result2 <- prob_tdist(df = 49, lb = -2.14)
p2a <- plot(result2) + theme_minimal() +
  geom_vline(aes(xintercept=-2.14), color="#099587") +
  geom_label(aes(x=-2.25, y=0.25, label="t=-2.14")) +
  geom_label(aes(x=0, y=0.075, label="p=0.981"), size=2) +
  geom_label(aes(x=-3, y=0.075, label="p=0.0187"), size=2) +
  guides(size="none") +
  labs(title="Result", x="t") + 
  xlim(-4,4)
library(patchwork)
p1a + p2a

Critical Values
P(X < -1.299) = 0.1
P(X > -1.299) = 0.9

Result: The p-value
P(X < -2.14) = 0.019
P(X > -2.14) = 0.981

t.test(cars$speed, mu=17, conf.level = 0.9, alternative="less")

## 
##  One Sample t-test
## 
## data:  cars$speed
## t = -2.1397, df = 49, p-value = 0.01869
## alternative hypothesis: true mean is less than 17
## 90 percent confidence interval:
##      -Inf 16.37143
## sample estimates:
## mean of x 
##      15.4

Case 3: A Greater Alternative

result <- prob_tdist(df = 49, plb = 0.9)
p1a <- plot(result, type = "probs") + theme_minimal() + geom_label(aes(x=1.299, y=0.25, label="t=1.299")) +
  geom_label(aes(x=0, y=0.075, label="p=0.9"), size=2) +
  geom_label(aes(x=3, y=0.075, label="p=0.1"), size=2) +
  labs(title="Critical Value") +
  xlim(-4,4)
result2 <- prob_tdist(df = 49, lb = -2.14)
p2a <- plot(result2) + theme_minimal() +
  geom_vline(aes(xintercept=-2.14), color="#099587") +
  geom_label(aes(x=-2.25, y=0.25, label="t=-2.14")) +
  geom_label(aes(x=0, y=0.075, label="p=0.981"), size=2) +
  geom_label(aes(x=-3, y=0.075, label="p=0.0187"), size=2) +
  guides(size="none") +
  labs(title="Result", x="t") + 
  xlim(-4,4)
library(patchwork)
p1a + p2a

Critical Values
P(X < -1.299) = 0.1
P(X > -1.299) = 0.9

Result: The p-value
P(X < -2.14) = 0.019
P(X > -2.14) = 0.981

t.test(cars$speed, mu=17, conf.level = 0.9, alternative="greater")

## 
##  One Sample t-test
## 
## data:  cars$speed
## t = -2.1397, df = 49, p-value = 0.9813
## alternative hypothesis: true mean is greater than 17
## 90 percent confidence interval:
##  14.42857      Inf
## sample estimates:
## mean of x 
##      15.4

result <- single_mean(
  cars, 
  var = "speed", 
  comp_value = 17, 
  alternative = "greater", 
  conf_lev = 0.9
)
summary(result)

## Single mean test
## Data      : cars 
## Variable  : speed 
## Confidence: 0.9 
## Null hyp. : the mean of speed = 17 
## Alt. hyp. : the mean of speed is > 17 
## 
##    mean  n n_missing    sd    se    me
##  15.400 50         0 5.288 0.748 1.254
## 
##  diff    se t.value p.value df    10% 100%  
##  -1.6 0.748   -2.14   0.981 49 14.429  Inf  
## 
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

# plot(result, plots = "hist", custom = FALSE)

Equations

First, doing the math by hand, I get:

\[ t = \frac{\overline{x} - \mu}{\frac{s}{\sqrt{n}}} = \frac{15.4 - 17}{\frac{5.29}{\sqrt{50}}} = -2.14 \]

Interpreting the result, the sample mean is 2.14 standard errors below the hypothetical mean of 17. Any \(t\) less than -1.299 would have been sufficient to reject the claim that \(\mu \geq 17\) and conclude that it must be less than 17 with 90% confidence.

I could also use the second manipulation above to study this in the original metric. We would accept [or fail to reject] the claim that \(\mu \geq 17\) if \(t \geq -1.299\). Let’s plug this in.

\[t \geq \frac{\overline{x} - \mu}{\frac{s}{\sqrt{n}}} \rightarrow \mu + t\left(\frac{s}{\sqrt{n}}\right) \geq \overline{x} \rightarrow 17 + (-1.299)\left(\frac{5.29}{\sqrt{50}}\right) \geq \overline{x} \rightarrow 16.03 \geq \overline{x}\]

Put in words, we could sustain the belief that \(\mu \geq 17\) with 90% confidence as long as we see a sample mean of 16.03 or greater given this standard deviation and sample size. Ours is 15.4 so we must reject the claim that \(\mu \geq 17\) with 90% confidence.

plot(17 + 0.7478*seq(-5,5, by=0.01), dt(seq(-5,5, by=0.01), df=49), xlab=expression(paste(mu)), main="measured in std. errors of the mean", ylab="Density", type="l")

The Confidence Interval

The confidence interval is always centered on the sample mean. Rearranging the equation above and solving for \(\mu\) given the \(t\) above, we get

\[ \mu = \overline{x} - t(\frac{s}{\sqrt{n}}) = 15.4 - (-1.299*\frac{5.29}{\sqrt{50}}) = 16.37143 \]

With 90% confidence, given this sample mean, the true value should be less than 16.37143.

The native t.test

t.test(cars$speed, conf.level = 0.9, alternative = "less", mu=17)

## 
##  One Sample t-test
## 
## data:  cars$speed
## t = -2.1397, df = 49, p-value = 0.01869
## alternative hypothesis: true mean is less than 17
## 90 percent confidence interval:
##      -Inf 16.37143
## sample estimates:
## mean of x 
##      15.4

Simplifying?

\[ t(\frac{s}{\sqrt{n}}) = \overline{x} - \mu \] can lead to either:

\[ \overline{x} - t(\frac{s}{\sqrt{n}}) = \mu \]

or

\[ \overline{x} = \mu + t(\frac{s}{\sqrt{n}}) \]

So a minus \(t\) will be below \(\mu\) but above \(\overline{x}\) and a positive \(t\) will be above \(\mu\) but below \(\overline{x}\).
1. An hypothesis test given \(\mu\) with an alternative that is less must then render an upper bound given \(\overline{x}\).
2. An hypothesis test given \(\mu\) with an alternative that is greater must then render a lower bound given \(\overline{x}\).

A graphical representation

Given a sample size \(n\), some unknown constant \(\mu\) and satisfaction of Lindeberg’s condition, the sampling distribution of the sample mean follows a \(t\) distribution with degrees of freedom \(n-1\). To render a graphical representation, let’s arbitrarily set n to 50, as in the above example. Here is a plot.

plot(seq(-5,5, by=0.01), dt(seq(-5,5, by=0.01), df=49), xlab=expression(paste("x-bar -",mu," (measured in std. errors of the mean)", sep="")), ylab="Density", type="l")

plot(seq(-5,5, by=0.01)*0.7478, dt(seq(-5,5, by=0.01), df=49), xlab=expression(paste("x-bar -",mu," (measured in mph)", sep="")), ylab="Density", type="l")

plot(x=17+seq(-5,5, by=0.01)*0.7478, dt(seq(-5,5, by=0.01), df=49), xlab=expression(paste(mu," (measured in mph)", sep="")), ylab="Density", type="l")
abline(v=17, col="red")
polygon(x = c(17+seq(0,5, by=0.01)*0.7478, 21), y = c(0, dt(seq(0,5, by=0.01), df=49)), col = "red")

plot(x=17+seq(-5,5, by=0.01)*0.7478, dt(seq(-5,5, by=0.01), df=49), xlab=expression(paste("x-bar -",mu," (measured in mph)", sep="")), ylab="Density", type="l")
abline(v=17, col="red")
abline(v=15.4, col="blue")
polygon(x = c(17+seq(0,5, by=0.01)*0.7478, 21), y = c(0, dt(seq(0,5, by=0.01), df=49)), col = "red")
polygon(x = c(12, 17+seq(-5,-2.14, by=0.01)*0.7478), y = c(dt(seq(-5,-2.14, by=0.01), df=49), 0), col = "blue")
abline(h=0, col="black")
abline(v=17 + qt(0.1, df=49)*0.7874, col="black", lty=3)

The Confidence Interval

Let’s take the sample mean as the center and work out a confidence interval at 90%. It’s exactly the 16.37143 gives above.

plot(x=15.4+seq(-5,5, by=0.01)*0.7478, dt(seq(-5,5, by=0.01), df=49), xlab=expression(paste(mu," | x-bar (measured in mph)", sep="")), ylab="Density", type="l")
abline(v=15.4, col="blue")
abline(v=15.4 - qt(0.1, df=49)*0.7478, col="black", lty=3)
polygon(x = c(11, 15.4+seq(-5,1.3, by=0.01)*0.7478), y = c(dt(seq(-5,1.3, by=0.01), df=49), 0), col = "blue")

As an aside, 17 has exactly 0.01869 probability above it shown in orange.

plot(x=15.4+seq(-5,5, by=0.01)*0.7478, dt(seq(-5,5, by=0.01), df=49), xlab=expression(paste(mu," | x-bar (measured in mph)", sep="")), ylab="Density", type="l")
abline(v=15.4, col="blue")
abline(v=15.4 - qt(0.1, df=49)*0.7478, col="black", lty=3)
polygon(x = c(11, 15.4+seq(-5,1.3, by=0.01)*0.7478), y = c(dt(seq(-5,1.3, by=0.01), df=49), 0), col = "blue")
polygon(x = c(15.4+seq(2.14,5, by=0.01)*0.7478, 17), y = c(dt(seq(2.14,5, by=0.01), df=49), 0), col = "orange")

Margin Figures

Images and graphics play an integral role in Tufte’s work. To place figures in the margin you can use the knitr chunk option fig.margin = TRUE. For example:

MPG vs horsepower, colored by transmission.

library(ggplot2)
mtcars2 <- mtcars
mtcars2$am <- factor(
  mtcars$am, labels = c('automatic', 'manual')
)
ggplot(mtcars2, aes(hp, mpg, color = am)) +
  geom_point() + geom_smooth() +
  theme(legend.position = 'bottom')

Note the use of the fig.cap chunk option to provide a figure caption. You can adjust the proportions of figures using the fig.width and fig.height chunk options. These are specified in inches, and will be automatically scaled down to fit within the handout margin.

Arbitrary Margin Content

In fact, you can include anything in the margin using the knitr engine named marginfigure. Unlike R code chunks ```{r}, you write a chunk starting with ```{marginfigure} instead, then put the content in the chunk. See an example on the right about the first fundamental theorem of calculus.

⊕We know from the first fundamental theorem of calculus that for \(x\) in \([a, b]\): \[\frac{d}{dx}\left( \int_{a}^{x} f(u)\,du\right)=f(x).\]

For the sake of portability between LaTeX and HTML, you should keep the margin content as simple as possible (syntax-wise) in the marginefigure blocks. You may use simple Markdown syntax like **bold** and _italic_ text, but please refrain from using footnotes, citations, or block-level elements (e.g. blockquotes and lists) there.

Note: if you set echo = FALSE in your global chunk options, you will have to add echo = TRUE to the chunk to display a margin figure, for example ```{marginfigure, echo = TRUE}.

Full Width Figures

You can arrange for figures to span across the entire page by using the chunk option fig.fullwidth = TRUE.

ggplot(diamonds, aes(carat, price)) + geom_smooth() +
  facet_grid(~ cut)

A full width figure.

Other chunk options related to figures can still be used, such as fig.width, fig.cap, out.width, and so on. For full width figures, usually fig.width is large and fig.height is small. In the above example, the plot size is \(10 \times 2\).

Arbitrary Full Width Content

Any content can span to the full width of the page. This feature requires Pandoc 2.0 or above. All you need is to put your content in a fenced Div with the class fullwidth, e.g.,

::: {.fullwidth}
Any _full width_ content here.
:::

Below is an example:

Main Column Figures

Besides margin and full width figures, you can of course also include figures constrained to the main column. This is the default type of figures in the LaTeX/HTML output.

ggplot(diamonds, aes(cut, price)) + geom_boxplot()

A figure in the main column.